Storage Capacity of the Tilinglike Learning Algorithm

The storage capacity of an incremental learning algorithm for the parity machine, the Tilinglike Learning Algorithm, is analytically determined in the limit of a large number of hidden perceptrons. Different learning rules for the simple perceptron are investigated. The usual Gardner-Derrida one leads to a storage capacity close to the upper bound, which is independent of the learning algorithm considered.Comment: Proceedings of the Conference Disordered and Complex Systems, King's College, London, July 2000. 6 pages, 1 figure, uses aipproc.st

A Model for Scaling in Firms' Size and Growth Rate Distribution

We introduce a simple agent-based model which allows us to analyze three stylized facts: a fat-tailed size distribution of companies, a `tent-shaped' growth rate distribution, the scaling relation of the growth rate variance with firm size, and the causality between them. This is achieved under the simple hypothesis that firms compete for a scarce quantity (either aggregate demand or workforce) which is allocated probabilistically. The model allows us to relate size and growth rate distributions. We compare the results of our model to simulations with other scaling relationships, and to similar models and relate it to existing theory. Effects arising from binning data are discussed.Comment: accepted for publication in Physica

Learning curves for Soft Margin Classifiers

Typical learning curves for Soft Margin Classifiers (SMCs) learning both realizable and unrealizable tasks are determined using the tools of Statistical Mechanics. We derive the analytical behaviour of the learning curves in the regimes of small and large training sets. The generalization errors present different decay laws towards the asymptotic values as a function of the training set size, depending on general geometrical characteristics of the rule to be learned. Optimal generalization curves are deduced through a fine tuning of the hyperparameter controlling the trade-off between the error and the regularization terms in the cost function. Even if the task is realizable, the optimal performance of the SMC is better than that of a hard margin Support Vector Machine (SVM) learning the same rule, and is very close to that of the Bayesian classifier.Comment: 26 pages, 10 figure

Rigorous Bounds to Retarded Learning

We show that the lower bound to the critical fraction of data needed to infer (learn) the orientation of the anisotropy axis of a probability distribution, determined by Herschkowitz and Opper [Phys.Rev.Lett. 86, 2174 (2001)], is not always valid. If there is some structure in the data along the anisotropy axis, their analysis is incorrect, and learning is possible with much less data points.Comment: 1 page, 1 figure. Comment accepted for publication in Physical Review Letter

Discrete Choices under Social Influence: Generic Properties

We consider a model of socially interacting individuals that make a binary choice in a context of positive additive endogenous externalities. It encompasses as particular cases several models from the sociology and economics literature. We extend previous results to the case of a general distribution of idiosyncratic preferences, called here Idiosyncratic Willingnesses to Pay (IWP). Positive additive externalities yield a family of inverse demand curves that include the classical downward sloping ones but also new ones with non constant convexity. When j, the ratio of the social influence strength to the standard deviation of the IWP distribution, is small enough, the inverse demand is a classical monotonic (decreasing) function of the adoption rate. Even if the IWP distribution is mono-modal, there is a critical value of j above which the inverse demand is non monotonic, decreasing for small and high adoption rates, but increasing within some intermediate range. Depending on the price there are thus either one or two equilibria. Beyond this first result, we exhibit the generic properties of the boundaries limiting the regions where the system presents different types of equilibria (unique or multiple). These properties are shown to depend only on qualitative features of the IWP distribution: modality (number of maxima), smoothness and type of support (compact or infinite). The main results are summarized as phase diagrams in the space of the model parameters, on which the regions of multiple equilibria are precisely delimited.Comment: 42 pages, 15 figure